All that Glitters is not Gold: A Comparison Between Generative and Traditional AI Chatbots in Education

This study delves into the transformative impact of Artificial Intelligence (AI) in education, specifically focusing on the comparison between traditional AI, hybrid, and generative AI-only chatbots. Partnering with Rocky.ai, a prominent European AI Coaching chatbot company, we explore students' acceptance and intention to use these chatbots for enhancing soft skills. Our research questions address the factors influencing the intention to use traditional AI and hybrid chatbots compared to generative AI-only counterparts. Employing structural equation modeling and extending the Unified Theory of Acceptance and Use of Technology model, we reveal substantial differences in user perceptions. Notably, although hybrid chatbots are perceived riskier than traditional AI chatbots, present higher Behavioral Intention to use than generative AI-only chatbots. Finally, we identified key mediators (e.g., Perceived Trust and Performance Expectancy), shedding light on critical factors influencing users' intention to engage with these technological tools. This research contributes to the field by offering insights into the distinctions between chatbot types. The study advocates for a thoughtful approach to adopting generative AI solutions, recognizing the diverse capabilities and user perceptions associated with different chatbot models. Among the practical contributions we offer actionable insights for educators and practitioners seeking to integrate AI chatbots into educational settings.